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ABSTRACT 

The research reported in this paper investigated the 
efficacy of the use of comprehensive computer-based instruction (CBI) 
for providing basic skills remediation to educationally disadvantaged 
student populations. Thirteen CBI programs placed in 26 elementary 
and secondary schools throughout the New York City school system were 
evaluated during the 1987-88 school year. Results reveal that CBI 
programs can indeed be an effective means for 'ivering such 
instruction, that they can be as effective in providing instruction 
in leading as they are in providing mathematics instruction to 
educationally disadvantaged students, and that within that population 
an inverse relationship exists between instructional level and 
achievement gains resulting from involvement with CBI. The 
differential effectiveness of differing programs was also suggested 
in the findings, interviews with participating students and teachers 
indicate that four features of CBI make it particularly useful to 
educationally disadvantaged students — CBI is perceived by students 
as less threatening than traditional classroom instruction, it 
provides extensive drill and practice exercises, it typically 
provides individualized diagnostics, and CBI programs provide 
students with greater academic support. Results of data analyses are 
reported in 18 tables. (14 references) (Author) 
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Abstract 



The research reported in this paper investigated the efficacy of the use of 
comprehensive computer-based instruction for providing basic skills 
remediation to educationally disadvantaged student populations. Thirteen 
CBI programs placed in twenty-six elementary and secondary schools 
throughout the New York City school system were evaluated during the 
1987/88 school year. Results reveal that CBI programs can indeed be an 
effective means for delivering such instruction, that they can be as effective 
in providing instruction in reading as they are in providing mathematics 
instruction to educationally disadvantaged students, and that within that 
population an inverse relationship exists between instructional level and 
achievement gains resulting from involvement with CBI. The differential 
effectiveness of differing programs was also suggested in the findings. 
Interviews with participating students and teachers indicate that four 
features of CBI make it particularly useful to educationally disadvantaged 
students - CBI is perceived by students as less threatening than traditional 
classroom instruction, it provides extensive drill and practice exercises, it 
typically provides individualized diagnostics, and CBI programs provide 
students with greater academic support. 
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Many authors have argued that comprehensive computer based instruction 
might best be used for delivering basic skills instruction to educationally 
disadvantaged students (Niemiec & Walberg, 1987). Not only is there a critical need 
for some means of addressing persistent functional illiteracy among an unacceptable 
percentage of the American population (National Commission on Excellence in 
Education, 1983; Commission on Reading, 1986; Kirsch & Jungeblut, 1986; National 
Assessment of Educational Progress, 1988), but research suggests that educationally 
disadvantaged students benefit most from the use of CBI systems (Jamison, Suppes & 
Wells, 1970; Chambers & Sprecher, 1980; Schmidt, Weinstein, Niemiec & Walberg, 
1985; Niemiec & Walberg, 1987). Indications are also that students learn more quickly 
from CBI programs than from regular classroom instruction (Ragosta, 1982; Kulik, Kulik 
& Bangert-Drowns, 1985; Bangert-Drowns, Kulik & Kulik, 1985) and that their 
motivation for and attention to needed drill and practice is improved by the use of 
computer delivery (Electronic Learning Laboratory, 1982; White, 1986; Niemiec & 
Walberg, 1987). Such factors may be particularly important to educationally 
disadvantaged students. To date, however, no large scale, systematic investigations 
of the use of CBI for basic skills remediation among educationally disadvantaged 
student populations have been conducted. 

Such is the work of the Computer Pilot Program of the Division of Computer 
Information Services of the New York City Board of Education. The Computer Pilot 
Program was designed to investigate the educational uses of computers, and to 
explore the efficacy of their use with New York City's educationally disadvantaged 
student population. The research reported in this paper is concerned with the 
statistical analyses of the effects of thirteen CBI programs on the reading and 
mathematics performance of educationally disadvantaged students in the third through 
tenth grades. It is unique in that it examines within a single study ihe instructional 
effects of a large sample of comprehensive CBI systems. It also looks at a very large 
educationally disadvantaged student population in many, very real, inner-city school 
settings. 

Methodology 

The Computer Pilot Program was funded by the Division of Computer 
Information Services of the New York City Board of Education In conjunction with the 



vendors (in the 1987/88 school year) of thirteen computer-based instructional systems 
- Autoskllls, CCC, CCP, CNS, Degem, ESC, ideal, PALS, PC Class, Plato, 
Prescription Learning, Wasatch, and Wicat Vendors placed their systems in 
appropriate schools, and offered staff development, equipment maintenance, and 
support at little or no cost for the evaluation period. Individual schools were 
responsible for staffing the programs and providing time for staff development. The 
Division of Computer Information Sciences provided additional equipment, technical 
assistance, and coordination among groups involved in the Computer Pilot Program. 
Evaluation of the Computer Pilot Program was carried out by the Office of Educational 
Assessment of the New York City Board of Education. 

The goals of the Computer Pilot Program are to identify comprehensive 
computer-based instructional programs which can be effective in increasing the 
academic performance, attendance, and positive attitudes of educationally 
disadvantaged students, and to isolate the implementation factors significantly 
influencing program and/or implementation effectiveness. 

During the 1987/88 school year, comprehensive CBI programs were 
evaluated in ten elementary, seven junior high or intermediate schools, and nine hiph 
schools throughout New York City. Most school sites were chosen by the Chancellor, 
and at each a program coordinator was selected by the school or the school district to 
be responsible for the daily operation of the program. Each school was also 
responsible for salecting a target group of students in need of remediation in basic 
reading and/or mathematics, and for scheduling that group in compliance w; r the 
stated needs of the particular systems. During the 1987/88 school year the citywide 
test scores of 1,734 students were included in the analysis of improvements in reading 
performance, and the scores of 1,361 students were used to analyze mathematics 
performance improvements. 

Tests used were the Degrees of Reading Power (DRP) for reading 
achievement and the Metropolitan Achievement Test (MAT) for mathematics 
performance. Each student's 1987 and 1988 mid-instructional scores compared using 
several statistical analysis. Only students for whom both 1987 and 1988 scores were 
available were included in the study. Students In grades eleven and twelve do not 
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take DRP's, and students in grades nine through twelve do not take MAT'S, thus 
participating students in those grades were not included in the study. Student 
groupings containing less than five subjects were also omitted from the analyses in 
which they occurred. 

Matched T-tests were used to test for significant differences between students 
1987 and 1988 DRP and MAT scores, and effect sizes for the mean differences 
between these generated. Effect sizes were calculated by dividing mean differences 
by their standard deviations. Effect sizes of one or greater thus indicate performance 
increases of a full standard deviation or more. Matched T-tests and effect sizes were 
used to compare both reading and mathematics scores for computer systems 
(programs) by grade levels by individual schools, for grade clusters (ie. elementary, 
junior high, high school) by student categories (ie. special education, Chapter 1, 
bilingual, remedial, general), and for grade clusters by student categories by CBI 
programs. 

In addition, analyses of covariance were used to compare 1988 mean scores 
between various student groupings (ie. grade clusters, student categories, computer 
systems) to test for significant differences among these. 1987 scores were used as 
covariates to control for individual differences. ANCOVA's were also used to test for 
significant interactions between grade clusters and computer systems, and between 
grade clusters, student categories, and computer systems. 

Results 

Table 1 shows the results of matched T-test comparisons broken down by 
systems and schools and grades for reading scores. Table 2 shows the results of 
matched T-test comparisons broken down by systems and schools and grades for 
mathematics scores. These results show that in general, students participating in the 
Computer Pilot Program showed significant increases in both reading and 
mathematics performance. An overall effect size of 0.8 was found across in reading 
improvements and an overall effect size of 0.9 was found across in mathematics 
improvements, indicating that, in general, such increases were educationally 
meaningful. 
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Notice that these findings indicate that the CBI programs tested were, in 
general equally effective in producing increases in students' reading and mathematics 
performance. Such results argue against commonly held notions (White, 1986; 
Niemiec & Walberg, 1987) that CBI is a more effective means for delivering 
mathematics instruction than for delivering reading instruction, at least among the 
educationally disadvantaged student population tested. 

Notice also that effect sizes decrease with advancing grade levels. Effect 
sizes in reading break down to 1.1 for elementary school students, 0.7 for junior high 
students, and 0.3 for high school students. In mathematics, effect sizes break down to 
1 .2 for elementary students and 0.4 for junior high students. To test for the statistical 
significance of this effect, analyses of covariance were used to compare the 1988 
scores of students in three grade clusters - elementary, junior high, and high school - 
for reading, and two grade clusters - elementary and junior high school - for 
mathematics, using 1987 scores as a covariate to control for individual differences. 
Table 3 shows the results of the analysis of reading scores; Table 4 shows the the 
results of the analysis of mathematics scores. 



Table 3 
ANCQVA TAB! F * 

1968 CHywide Degrees of Heading Rower (DRP) Sco res bv Grade austere 

SS_ DJF MS EE 

W/!N CELL 84244.76 1583 53.22 

REGRESSION 148676.19 1 148676.19 2793.70 000 

GRADCLUST 1375.96 2 687.98 12.93 .000 

*1987 DRP reading scores as covariate 
Table 4 

AMCnVATApiF* 

1988 Safe! MrtmnolIMn Achievement Tp sI /MAT) Math Score hy g^fa CjugBB 

SS OF MS EE 

W/1N CELL 587073.01 1348 435.51 

REGRESSION 1481310.94 1 1481310.94 3401 29 000 

GRADCLUST 53823.15 1 53823.15 123.59 .000 



*1987 MAT math scores as covariate 
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The results reveal significant differences between grade clusters in both reading (F-| t 
1683 a 12.93, p < .01) and mathematics (F1J348 s 123.59, p < .01). These findings 
corroborate Kulik's (1981) finding of an inverse relationship between students' instructional 
level and performance gains resulting from CBI use. They are particularly meaningful 
because they corroborate the results of his meta-analysis within a single study and with 
respect to an educationally disadvantaged student population. 

To test the parameters of this relationship, several comparisons were made involving 
student categories. Analysis of covariance were used to compare 1988 mean scores in both 
reading and mathematics among students in five student categories - special education, 
Chapter 1, bilingual, remedial, and general education - which were understood as 
representing, in that order, increasingly advanced levels of instruction. 1987 mean scores 
were used as the covariate. The results of these analyses are given in Tables 5 and 6. They 
indicate significant differences in performance gain between student categories in both 
reading (F4J581 = 11.40, p < .01) and mathematics ^4,1345 ■ 24,74, p < .01). Table 7 
shows effect sizes by grade clusters and student categories for reading gains, and Table 8 
shows effect sizes by grade clusters and student categories for gains in mathematics. Notice 
that, especially within the lower grades, an inverse relationship between instructional level 
and achievement can be quite clearly distinguished. Special education students benefited 
most from CBi use at all grade levels, and remedial and general education students benefited 
least at most grade levels. The inverse relationship seems clearer, however, among students 
in the lower grades. 

Table 5 
ANCOVA TABLE* 

1968 Crtywkte Degrees of Reading Power (DRP) Scores by Student Categories 



53. 

W/INCELL 83221.31 
REGRESSION 176310.06 
STUDCAT 2399.42 



DE MS EE 

1581 52.64 

1 176310.06 3349.46 .000 

4 599.85 11.40 .000 

*1987 DRP reading scores as covariate 
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Table 6 
ANCOVA TABLE * 

1 968 Seated Metropolitan Achievement Test ( MAT) Math Scores by Student r^ategnries 



SS DF MS F P 

W/1N CELL 596979.00 1345 443 85 

REGRESSION 1180858.49 1 1180858.49 2660.49 000 

STUDCAT 43917.16 4 10979.29 24.74 '.000 



"1987 MAT math scores as covariate 



Table 7 
EFFECT SIZES 
1968 PRP Scores bv Grade dusters and Stud ent Categ ories 
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0.8 


JUNIOR HIGH: 












Special Ed 


19 


43.1 


57.3 


14.2** 


4.8 


3.0 


Chapter 1 


266 


51.6 


56.5 


4.9* 


7.4 


0.7 


Remedial 


241 


54.9 


59.8 


4.9* 


6.9 


0.7 


General 


44 


59.6 


62.3 


2.7 


9.7 


0.3 


HIGH SCHOOL- 














Remedial 


21 


57.9 


61.3 


3.4* 


4.9 


0.7 


Chapter 1 


296 


56.4 


58.6 


2.2 


6.8 


0.3 



Table 8 

EFFECT SIZES 

1968 >4AT Math Scores by Grade Ousters and Student Cateoories 



STUDCAT 


N 


1907 


1968 


MEAN DIF 


SD 


IS 


ELEMENTARY: 










Special Ed 


11 


567.5 


604.1 


36.5* 
30.0* 
31.4* 
28.4* 
23.2* 


25.6 


1.4 


Chapter 1 


371 


565.8 


595.8 


22.5 


1.3 


Bilingual 


58 


578.9 


610.4 


26.3 


1.2 


General 


296 


621.1 


649.5 


25.3 


1.1 


Remedial 


128 


582.9 


606.1 


23.1 


1.0 


JUNIOR HIGH: 








Special Ed 


17 


602.1 


621.5 


19.4** 
9.6* 
6.7* 


22.9 


0.8 


Remedial 


202 


632.4 


642.0 


16.7 


0.6 


Chapter 1 


242 


619.9 


626.7 


17.1 


0.4 


General 


41 


653.4 


648.2 


-5.2 


25.2 


-0.2 



* - significant difference at p < .01 level 
**- eignifioant difforono© at p * .06 lev© 



To test for the significance of these seemingly different patterns of student gains, 
analyses of covariance were used to compare 1988 mean scores in both reading and 
mathematics among students by grade clusters and student categories using 1987 mean 
scores as the covariate (Tables 9 and 10): The results indeed reveal significant interactions 
between grade clusters and student categories in both reading (F5J574 = 2.92. p < .05) and 
mathematics (F4J240 = 86 6. P < -01), indicating differential patterns of achievement among 
student categories within differing grade clusters. Effect size data indicates that the major 
source of such differences involves the differential ranking of Chapter 1 and remedial 
students at differing grade levels. In general, however, findings show that an inverse 
relationship between Instructional level and achievement gains resulting from CBI use holds 
not only between grade levels but between student categories within grade levels. 



Table 9 
ANCOVA TABLE* 
1968 DRP Scores by Grade Cluster and Student Catg ory 





ss_ 


DF 


MS 


F 


P 


W/IN CELL 


82162.10 


1574 


52.20 






REGRESSION 


102165.11 


1 


1021165.11 


1957.20 


.000 


GRDCLUST 


190.46 


2 


95.23 


1.82 


.162 


STUOCAT 


189.18 


4 


47.30 


.91 


.460 


GCXSC 


761.41 


5 


152.28 


2.92 


.013 



*1987 DRP reading scores as covariate 

Table 10 
ANCOVA TABLE* 
1988 Scaled MAT Math Scores by Grade duster and Student Cat gory 





SS 


DF 


MS 


F 


B 


W/IN CELL 


560357.78 


1340 


131.63 






REGRESSION 


975156.66 


1 


975156.66 


2331.92 


.000 


GRDCLUST 


1350.77 


1 


1350.77 


3.23 


.073 


STUDCAT 


1878.54 


4 


4S9.64 


1.12 


.344 


GCXSC 


14525.65 


4 


3631.41 


8.68 


.000 



*1987 MAT math scores as covariate 



Table 11 shows the analysis of covariance comparing 1988 mean scores in reading 
between students using differing CBI programs, and Table 12 shows the analysis of 
covariance comparing 1988 mean scores in mathematics between students using differing 
CBI programs. In both cases, 1987 mean scores were used as the covariate to control for 

10 
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indiv : dual differences. The results reveal the differential effects of particular computer systems 
as indicated by significant differences in students' performance increases between computer 
systems in both reading (Fi2,1716 * 11.98, p < .01), and mathematics (F8J341 * 13.00, p < 
.01). Because, however, certain CBI programs were only used at particular grade levels, and, 
as we have seen, particular grade svels experienced significantly greater performance 
increases than others, seeming differences Jjetween system* t" be the result of differences 
between grade clusters rather than real differences between systems. For example, CCC and 
Wicat, the programs with the seemingly greatest effects, were only used at the elementary 
level, the grade level exhibiting the largest effect sizes. On the other hand, these programs 
were also most effective within that level in both reading ancfmathematics, an indication that 
the programs themselves were at least partially responsible for their successes. 

Table 11 

19B8 Cttywkte Ptegmpg d Reading Power fURPI Scores by Computer 



SS QE MS E P. 

W/INCELL 172496.93 1716 100.52 

REGRESSION 168533.29 1 158533.29 1577.09 .000 

COMPSYS 1 4457.1S 12 1204.76 11.98 .000 

"1987 DRP r-^'ng scores as covariate 

Table 12 
ANCOVA TABLE * 

19Q8Scaled Metropolitan Achka/ementTest fMAT> Math Score, by Computer Systems 

SS DF MS F P 

W/INCELL 594766.97 1341 443.52 

REGRESSION 1169511.28 1 1169511.30 2636.86 .000 

COMPSYS 46129.19 8 5766.15 13.00 .000 

*1987 MAT math scores as covariate 

To examine this issue in greater detail, the differential effectiveness of particular CBI 
systems within grade clusters was tested using analyses of covariance with 1987 mean 
scores as covariates (Tables 15 and 16). A significant interaction between computer 
systems and grade clusters in reading (^1557 » 2.80, p < .01), but not in mathematics 
( F 3,1337 " -73, p > .10) was found, indicating that particular CBI programs were more 
effective at certain grade levels than at others for delivering reading, but not mathematics, 

ERIC ?■ 



instruction. In particular, effect size data reveal that the PC Class and Autoskills systems 
were among the least effective of the programs tested at the elementary level for the delivery 
of reading instruction, but among the most effective with junior high students. Because 
these results run counter to the general inverse relationship between achievement gains 
and instructional level, they should be seriously considered. 

Table 15 
ANCOVA TABLE* 
1968 DRP Scores by Grade Cluster anc Computer Systems 





SS 


DF 


MS 


F 


P 


W/1N CELL 


80878.48 


1567 


51.61 






REGRESSION 


100544.20 


1 


100544.20 


1948.02 


.000 


GRDCLUST 


0.0 


2 


.00 


.00 


1.000 


COMPSYS 


1791.56 


12 


149.30 


2.89 


.001 


GCXCS 


1010.28 


7 


144.33 


2.80 


.007 



*1987 DRP reading scores as covariate 
ANCOVA TABLE * 

1988 Seated MAT Math Scores by G * ade Ouster and Computer Systems 





SS 


DF 


MS 


F 


P 


W/INCELL 


555773.43 


1337 


415.69 






REGRESSION 


1125467.70 


1 


1125467.70 


2707.49 


.000 


GRDCLUST 


2320.99 


1 


2320.99 


5.58 


.018 


COMPSYS 


28043.29 


8 


3505.41 


8.43 


.000 


GCXCS 


909.22 


3 


303.07 


.73 


.535 



*1987 MAT math scores as covariate 



Tables 17 and 18 show mean differences and effect sizes for computer systems by 
student categories and grade clusters. Notice that in reading (Table 17), students using the 
CCC, Prescription Learning, and Wicat systems evidenced the greatest effect sizes in all 
student categories at the elementary level, p.nd that stuoents using the Prescription Learning, 
Autoskills, and PC Class systems evidenced the greatest effect sizes across student 
categories at the junior high school level. Ideal and Plato were most effective across 
categories of high school students. The differential effectiveness of the PC Class and 
Autoskills programs for varying grade levels is also evident in mathematics (Table 18), 
students using the CCC, Prescription Learning, and Wicat programs again evidenced the 
greatest effect sizes at the elementary level, and the use of Perscription Learning, PC Class, 
and Degem resulted in the greatest performance increases at the junior high level. 




Tat* 17 

EFFECT SIZES 



QVQTPli 
OTOl CM 


9 WWIV». ViW 
tJ 

a 


1987 


my-— "BTTypy 
1988 


MEANDIF 


SD 


ES 


ELEMENTARY* 






SPFC1AL FH- 
wrtvlAU lUl 














Prescription L 


e 


3A ft 


CO C 

OO.O 


4 C 7* 

18.7 


e a 

5.0 


3.3 


CHAPTER 1- 

\0 1 WW* 1 • 11 If 










ccc 

www 


Q 


1 Q a 


32.7 


13.1 


6.4 


2.0 


Wicat 


45 


34 3 

W7.W 


44 *> 
44.0 


1 n 9* 

1 U.2 


0.0 


4 c 

1.5 


Autoskills 


71 


20 6 


9A 7 


A 9* 

0.2 


A 1 

0.1 


4 0 

1.0 


PrescriDtion L 


A 

w 


iq n 


wi7 a 
47.0 


0 A* 

8.0 


A A 

6.4 


1.3 


ESC 


17 


97 1 


36.8 


9.5 


8.4 


1.1 


Lscryci 1 1 


1 in 

1 OU 


10 ^ 
08. 4 


46.1 


7.7 


8.0 


1.0 


BILINGUAL 












CCC 


A 

w 




wlA O 

40.2 


A A A * 

20.0 


8.0 


2.5 


Decern 


10 
i \j 


40 A 


CA O 

50.3 


9.5* 


3.9 


2.4 


vvaSKKiwn 


91 


41.0 


53.0 


44 4 * 
1 1.4 


11.0 


1.0 


PC Class 


7 

* 


15 0 

w w.U 


411 7 
41 / 


A 7*** 

6.7 


O A 

8.2 


0.8 


GENERAL 










CCC 

www 




17 5 

Of .0 


49.0 


11.5* 


8.0 


1.4 


L/Cy CI 1 1 


I 1 4 


COO 

52.2 


59.1 


8.9 


8.8 


0.8 


YYBSCHwTi 


79 
/ 2 


57.5 


83.3 


5.8 


8.9 


0.8 


REMEDIAL 








nCtfviipMvii ^ 


97 


4o.9 


54.2 


4 A A* 

10.3 


8.4 


1.2 


Decern 


33 

WW 




C O A 

52.0 


7.1 


10.0 


0.7 


Wasatch 


42 


11 4 


36.1 


41 7* 

4.7 


7.2 


0.7 


JUNIOmiGH' 








qpfoial Fry 














PrescriDtion L 

I 1 W ilvl i W Uwl 1 L» 


1Q 


41 1 
40. 1 


C 7 A 

57.3 


4 4 
14.2 


4.8 


3.0 


CHAPTER 1 • 










9A 


it 7 C 


55.3 


7.8 


8.3 


1.2 




1 Oft 


51.7 


58.7 


4.9 


7.3 


0.7 


Deaem 


1 1 n 


59 C 


CA 7 

56.7 


A A* 

4.2 


7.8 


0.5 


REMEDIAL 












PC niAM 


1 HA 
1 UO 


C7 9 

Or. 2 


82.8 


5.8 


8.9 


0.8 


Aye 
UNO 


39 


54.7 


59.4 


4.7* 


7.0 


0.7 


w cyci 1 1 




02.4 


CA A 

56.6 


4.3 


8.9 


0.8 


GENERAL 












PC fleas 

r W WIAN 


o 
o 




coo 
58.8 


7 0* 

7.9 


5.7 


1.3 


CMS 
w n w 




Al 9 


000 

68.8 


5.5 


4.9 


1.1 


Wasatch 


19 

1 w 


AO 1 


C7 O 


A A 

-2.2 


4 A A 

12.2 


A A 

-0.2 


HIGHSCHOOL 












REMEDIAL 














Ideal 


1 1 

1 1 


5? A 

w£.D 


C4 A 
04.0 


41 0* 
4.3 


C A 

5.2 


A O 

0.8 


Degem 


7 


80.8 


w *t. w 


9 0 


4 9 


n c 


CHAPTER 1: 












Plato 


102 


57.3 


59.9 


2.8* 


7.0 


0.4 


Ideal 


11 


53.6 


56.8 


3.2 


8.8 


0.4 


PALS 


43 


50.3 


52.4 


2.1** 


8.1 


0.3 


Degem 


90 


80.7 


62.6 


1.9* 


8.8 


0.3 


CCP 


39 


54.1 


55.5 


1.4 


9.2 


0.2 



* - significant difference at p < .0 1 level 
'* - significant difference at p < .05 level 

* — Aigniftoent differeno# *t p - . 1 0 level 



T rbte .8 
EFFECT SIZES 

1968 MAT Math Scores; Grade Cluster by Student Category bv Computer System 



SYSTEM 
ELEMENTARY: 



1967 



1968 MEAN DIF SD 



PC 



Prescription L 


7 


585.1 


614.4 


29.3* 


19.0 


1.5 


CHAPTER 1: 














Prescription L 


7 


590.0 


630.9 


40.9* 


13.2 


3.1 


Wicat 


62 


552.4 


591.3 


138.9* 


17.8 


2.2 


Degem 


166 


581.8 


610.9 


29.2* 


22.6 


1.3 


CCC 


9 


551.4 


584.2 


32.8* 


25.6 


1.3 


ESC 


35 


553.4 


574.0 


20.6* 


23.3 


0.9 


BILINGUAL: 














CCC 


6 


548.8 


593.5 


44.7* 


21.3 


2.1 


Degem 


12 


589.4 


613.9 


24.5* 


17.3 


'1.4 


PC Class 


17 


550.9 


589.6 


38.6* 


33.0 


1.2 


Wasatch 


22 


602.4 


629.9 


27.5* - 


24.6 


1.1 


GENERAL: 














CCC 


50 


583.6 


625.4 


41.8* 


20.6 


2.0 


Wasatch 


72 


636.4 


661.2 


24.8* 


19.9 


1.2 


Degem 


174 


625.5 


651.6 


26.0* 


27.3 


1.0 


REMEDIAL: 














Perscription L 


28 


602.1 


634.9 


32.8* 


20.7 


1.6 


Degem 


42 


598.0 


625.7 


27.7* 


20.7 


1.3 


Wasatch 


57 


562.5 


577.2 


14.6* 


23.0 


0.6 



JUNIOR HIGH: 

SPECIAL ED: 



Prescription L 


17 


602.1 


621.5 


19.4** 


22.9 


0.8 


REMEDIAL: 














Degem 


88 


624.6 


637.7 


13.1* 


17.9 


0.7 


PC Class 


92 


639.7 


648.2 


8.5* 


14.3 


0.6 


CNS 


18 


636.0 


635.5 


-0.5 


17.9 


0.0 


CHAPTER 1: 














Wasatch 


127 


622.3 


630.0 


7.7* 


17.5 


0.4 


Degem 


88 


619.1 


623.9 


4.8* 


16.5 


0.3 


GENERAL: 














PC Class 


7 


623.4 


638.1 


14.7 


20.4 


0.7 


CNS 


17 


684.2 


675.5 


-8.7 


21.6 


-0.4 


Wasatch 


17 


635.0 


625.0 


-10.0 


27.5 


-0.4 



* - significant difference at p < .01 level 
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Discusion 

The results of the analyses indicate that CBI can indeed be an effective means for 
delivering remedial instruction in reading and mathematics to educationally disadvantaged 
students. Significant increases in both reading and mathematics scores were found for the 
majority of students participating in the Computer Pilot Program. That such increases were 
educationally meaningful is demonstrated by their effect sizes - 0.8 for reading and 0.9 for 
mathematics. These results are particularly meaningful given that they were culled from 
mostly start-up implementations operating within a very large and diverse educational system, 
and so were subject to all the constraints attendent upon such an effort. Included among such 
constraints were technical problems with particular implementations, a perceived lack of 
sufficient training among a majority of the staff participating in the program, minimal student 
involvement with many of the CBI programs, and, in certain cases, the newness of the 
programs themselves. An apparent lack of correlation between the achievement goals of the 
CBI programs tested and those of the City of New York, demonstrated in a regression analysis 
comparing students' system-embedded gains with their scores on citywide tests, may indicate 
another problem area. Indications are, then, that results could be even more positive as 
faculty and students adjust to the use of this new and different educational medium. 

Our findings also suggest that CBI was as effective a medium for the delivery of 
instruction in reading as it was for the delivery of mathematics instruction to the educationally 
disadvantaged student population we surveyed. While overall effect sizes for increases in 
mathematics were slightly higher than those for reading performance, a larger percentage of 
students advanced to grade level or above in reading than in mathematics. Likewise, 
differences seen in interactions between content areas and other variables (ie. grade clusters, 
student categories, and computer systems) tended to balance themselves out. Such results 
contradict commonly held beliefs asserting that CBi is a more effective medium for supporting 
mathematics instruction than for supporting instruction in reading (Niemiec & Walberg, 1987; 
White, 1986). They argue instead that CBI can be an effective means for providing remedial 
instruction in both areas, at least among educationally disadvantaged populations. 

Niemiec and Walberg (1987) suggest four reasons for CBI's particular effectiveness 
among educationally disadvantaged student populations. They contend that CBi is less 
threatening than instruction relying on classroom recitiation and that such lack of threat may 
be more meaningful to the educationally disadvantaged; that educationally disadvantaged 
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students may benefit more from the extensive drill and practice exercises typically offered in 
CBI programs; that the diagnostic procedures integral to most such programs may benefit 
educationally disadvantaged students more than others because they are more likely to need 
specific remediation; and that extra teaching resources, which tend to be more useful to the 
educationally disadvantaged because they need more academic support, may be available to 
students involved with CBI programs. Our interviews with students and teachers participating 
in the Computer Pilot Program support all four arguements. 

When asked how learning on computers differed from their regular classroom 
activities, the students we interviewed overwhelming responded that learning on computers 
was less threatening. "I don't have to talk," "I dont have to write," "My mistakes aren't 
embarrassing," " It doesn't talk back," and 'The computer doesn't yell at me" were typical of the 
answers we received. Not only teachers, but many students we interviewed stated that the 
large amount of practice they received was very helpful. "The computer calls on me every 
time," one student commented. "You have to think more," another said. Students interviewed 
also believed that the immediate and informative feedback typically accompanying CBI drill 
and practice exercises made them especially useful. Participating teachers we interviewed 
believed that the diagnostic procedures offered by the CBI programs we evaluated were 
among their most useful features. They thought that the feedback from such procedures had 
made them more aware of students' individual strengths and weakness thus helping them to 
target specific areas where extra remediation was needed. Finally, it is unquestionably the 
case that extra teaching resources necessarily accompany CBI programs because the 
programs themselves are an important extra resource. Students reported that the programs 
were responsive their needs. Teachers reported that they freed them, not only from routine 
bookeeping and disciplinary chores, but from general lecturing as well, thus allowing them to 
devote more individual attention to their students. In addition, the average student/teacher 
ratio in the program implementations we evaluated was 16/1, considerably lower than those 
common in regular classrooms in the New York City system. 

The results of our analyses also show significant differences between grade levels for 
both reading and mathematics performance gains. They thus support Kulik's (1981) assertion 
that an inverse relationship exists between instructional level and achievement gains 
resulting from CBI use. The effect sizes of student gains were greater for elementary students 
than for junior high students and greater for junior high students than for high school students. 
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Such findings indicate that the inverse relationship between grade level and CBI gains which 
Kulik found among general student populations is likewise true of educationally 
disadvantaged populations. 

The picture is not quite so clear when instructional level is considered in terms of 
student categories within grade levels. Indeed, significant interactions between grade 
clusters and student categories were found for both reading and mathematics. At the 
elementary level for both reading and mathematics, students at the lower levels of instruction 
(special education, Chapter"!, and bilingual) experienced greater achievement score 
increases than students at higher levels of instruction (remedial, general.) At the junior high 
level for reading, special education students showed the largest performance gains and 
general education students showed the smallest gains, but the gains evidenced by Chapter 1 
and remedial students were essentially the same. At the junior high level in mathematics, 
special education students again showed the greatest achievement gains and general 
education students, the least, but remedial students showed greater gains than those enrolled 
in Chapter 1 programs. The same is true of high school students' reading performance gains. 

It is not clear why the patterns of relationships among student categories thus seem to 
change with changes in grade levels. It may be that the characteristics of students making up 
the Chapter 1 and remedial categories change as grade levels increase. It is interesting to 
note in this regard that the numbers of students entering the Computer Pilot Program at or 
above grade level seem to decline with increasing grade levels. On the other hand, it may be 
that the needs of Chapter 1 and/or remedial students differ at differing grade levels. It could 
be, for example, that remedial students at the upper grade levels are particularly in need of 
the kinds of drill and practice abundantly available in most CBI programs, whereas at lower 
grade levels, remedial students are in greater need of instruction concerned with other things. 
In any case, the findings indicate that the use of CBI was most effective for special education 
students and least effective for general education students at all grade levels. CBI was more 
effective for Chapter 1 and bilingual than for remedial students at the elementary levels, but 
that it was more effective for remedial than for Chapter 1 students at the upper grade levels. In 
general, however, they indicate an inverse relationship between instructional level and 
performance gains resulting from CBI use even with grade level held constant. 
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The explanation for this Inverse relationship may simply be that CBI Is a more 
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effective delivery system for students at lower instructional levels. It is more likely that the 
design of current programs better supports the sort of content addressed at lower instructional 
levels. While the simple tutorial and drill and practice routines common to most systems seem 
particularly effective, not only for developing needed skills, but for diagnosing ski!! 
deficiencies at such levels, the problem solving skills inherent in higher level reading 
comprehension and mathematical problem solving may neither be so easily developed, nor 
the lack of such skills effectively diagnosed, by instructional models of this sort. In any case, 
the results clearly argue that the most effective use of CBI programs among the population 
tested was in the elementary grades and among special education students, that the least 
effective use of them was among high school and general education students. 

Results concerning the relative effectiveness of the differing CBI programs were 
difficult to analyze because the grade levels and student categories with which they were 
tested, variables which significantly effected program success, differed among differing 
programs. Differences in stating and implementation characteristics and in the larger 
educational environments found among participating schools also contributed to problems 
surrounding the analysis of program effectiveness, such that the extent to which the 
differential effectiveness of the varying programs may be attributed to factors external to these 
programs could not be determined. Participation in the CCC and Wicat programs, for 
example, resulted in increases in both reading and mathematics achievement scores that 
were among the largest found in the entire Computer Pilot Program. These programs, 
however, were tested only at the elementary level, thus while their effectiveness at this level 
was unquestionably demonstrated, their relative effectiveness was not necessarily shown. 
Likewise, some of the largest increases in the entire evaluation were shown by students using 
the Perscription Learning program, but these were ail special education students. 

Use of the Prescription Learning program, in fact, resulted in large achievement 
increases at both the elementary and junior high levels, indicating its effectiveness at both 
these levels. The PC Class program was another whose use resulted in large achievement 
gains in both reading and mathematics at the junior high level. Indeed, this program appears 
to have been more effective at this level than at the elementary level. As such a finding 
contradicts the general tendency for programs to be more effective at lower instructional 
levels, we might assume it says something about the program itself. The PC Class system, 
however, is a generic learning by objectives program which can utilize any IBM PC software. 




As it is likely that differing software programs were utilized in different implementations, it is 
more reasonable to assume that such performance variations resulted rather from differences 
in software and not some developmental differential in what is essentially a delivery system. 

Autoskills, which is just a reading program, was also very effective at the junior high 
but not at elementary level, and is most likely the source of the significant interaction found 
between Col programs and grade clusters for reading but not mathematics. This difference 
between grade levels seems also to have resulted from implementation differences rather 
than the program itself as Autoskills was used alone at the elementary level, but at the junior 
high level was integrated Into a total language arts program involving other Icon language 
arts software as well as off-computer reading and writing. 

At the high school level, for which only reading scores were available, the Ideal and 
Plato systems resulted in the greatest increases in student achievement. Because 
achievement gains were so low and mathematics scores were not available at this level, the 
effectiveness of these systems among high school populations was suggested but not clearly 
indicated in the findings. 



Certain systems seem to have been more effective in providing instruction in one 
content area than the other. Degem, for example, appears to have been reasonably effective 
for providing mathematics instruction to students at all grade levels, but not very effective in 
providing reading instruction at any level. On the other hand, use of the ESC system, which 
occurred only at the elementary level, resulted in greater gains in reading than in 
mathematics. It was not, however, a leader in either content area. The CNS system, which 
was used only at the junior high level seems to have been somewhat effective in providing 
reading instruction, but appears to have actually been counterproductive in the area cf 
mathematics. The Wasatch system appears to have only been effective in providing reading 
instruction, and then only at the elementary level. Wasatch, however, is not a 
learning-by-objectives program like most of the others, but rather a process-oriented program. 
Thus it may be that teachers need time to adjust to this type of program before they can make 
effective use of it. 



The effectiveness of the PALS and CCP systems which were designed specifically for 
high school students seems doubtful, especially considering the large amounts of student and 
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faculty time they demand. Because, however, they were only used with high school students 
whose achievement gains were generally low, it is not possible to reach a conclusion 
concerning these systems. 

Conclusions 

We can conclude, however, that CBI programs can be effectively used to deliver 
remedial instruction in reading and mathematics to educationally disadvantaged inner city 
students, that tN»y can be as effective for delivering instruction in reading as they are for 
delivering mathematics instruction to such populations, and that among such populations an 
inverse relationship exists between instructional level and CBI effectiveness as measured in 
terms of achievement gains. In addition, we can conclude that the CCC, Wicat, and 
Prescription Learning programs can be good vehicles for the delivery of remedial instruction 
in both reading and mathematics to elementary school students, and that the Presription 
Learning, Autoskills, and PC Class programs can be used effectively with junior high students. 
Indications are that the Ideal and Plato programs might be useful for providing high school 
students with basic skills remediation, and that the Oegem program may be well adapted for 
mathematics remediation at all instructional levels. Continued investigation of all programs at 
all levels is, of course, needed to better determine their relative efficacies. 

Other interesting questions needing further study include the long term effects of CBI 
use, issues of transfer from computer-based learning to other media, and changes in the 
structure of the overall educational environment resulting from large scale CBI use. Of 
particular interest are educational effects other than achievement gains resulting from 
involvement with CBI programs. Increased student motivation, for example, improved 
attendance, lower drop-out rates, and better student attitudes toward school and learning 
might be more beneficial to educationally disadvantaged students in the long ru.i than short 
term academic achievement gains. Student perceptions of increased control over their own 
learly clearly deserve further study. For the present, however, the pressing realities of the 
crisis in education, especially among educationally disadvantaged inner city populations, 
together with the demonstrated efficacy of at least certain CBI programs, argue quite strongly 
for their adoption for the delivery of basic skills remediation to such populations, particularly at 
lower levels of instruction. 
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